The following report contains model performance metrics for the NY City Hourly Probabilistic Residential Energy Demand Forecasting Pipeline.
Model performance was evaluated on both long-term and day-ahead forecasts. Evaluation was conducted using a holdout dataset of hourly energy
demand values between 2023-10-19 and 2024-05-23.
Long-Term Forecasting Performance
The following table contains performance metrics for the forecasting model compared with a yearly moving average baseline model.
|
MSE |
Weighted MSE |
MAE |
MAPE |
Wilcoxon Test p-value |
| Forecasting Pipeline |
132582.09 |
117777.65 |
274.49 |
0.05 |
NaN |
| Baseline Yearly MA |
618450.20 |
618450.20 |
635.02 |
0.14 |
<0.01 |
The following Plotly Figure helps to contextualize the forecasting model's performance by showing its predictions along with the actual energy demand
values. It also presents the 95% confidence interval bounds estimated by the forecasting model.
Day-Ahead Forecasting Performance
The following table contains performance metrics for the forecasting model compared with a yearly moving average baseline model.
|
MSE |
Weighted MSE |
MAE |
MAPE |
Wilcoxon Test p-value |
| Forecasting Pipeline |
908700.32 |
908575.29 |
820.12 |
0.16 |
NaN |
| Baseline Moving Avg |
350660.80 |
350660.80 |
512.26 |
0.10 |
1.0 |
| EIA Forecasts |
115596.45 |
NaN |
291.92 |
0.06 |
1.0 |
The following Plotly Figure helps to contextualize the forecasting model's performance by showing its predictions along with the actual energy demand
values. It also presents the 95% confidence interval bounds estimated by the forecasting model.
The following figure shows a permutation feature importance analysis.